% http://aij.ijcai.org/competition-section

%%This is a very basic article template.
%%There is just one section and two subsections.
\documentclass[]{elsarticle}

\usepackage[hyphens]{url}
\usepackage[hidelinks]{hyperref}
%\usepackage{url}
\usepackage{listings}
\usepackage[usenames,dvipsnames]{color}
%\usepackage{color}
\usepackage{csquotes}
\usepackage{dsfont}
\usepackage{caption}
\usepackage{subfig}
\usepackage{microtype}

\usepackage{booktabs}
\usepackage{pbox}
\usepackage{soul}

\biboptions{numbers,sort&compress}

\usepackage{tikz}
\usetikzlibrary{shapes.geometric}
\usetikzlibrary{positioning,shapes,shadows,arrows,calc}
\usetikzlibrary{intersections}
\pgfdeclarelayer{background}
\pgfdeclarelayer{foreground}
\pgfsetlayers{background,main,foreground}

\newcommand{\note}[2]{
	\noindent
	\vspace{0.25cm}
	\fcolorbox{Red}{Apricot}{\parbox{0.99\textwidth}{\textbf{NOTE (#1):\\} #2}}
	\vspace{0.25cm}
}
\newcommand{\emcite}[1]{\citeauthor{#1} (\citeyear{#1})}
% Uncomment the following line to hide notes.
%\renewcommand{\note}[2]{}

% \newcommand{\hh}[1]{\textcolor{blue}{#1}}
% \newcommand{\fh}[1]{\textcolor{green}{#1}}
% \newcommand{\pk}[1]{\textcolor{cyan}{#1}}
\newcommand{\hh}[1]{#1}
\newcommand{\fh}[1]{#1}
\newcommand{\pk}[1]{#1}


% Commands for theorems, lemmas, definitions, etc.
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{amssymb}
\newtheorem{thm}{Theorem}%[section]
\newtheorem{lem}[thm]{Lemma}
\newtheorem{prop}[thm]{Proposition}
\newtheorem{cor}[thm]{Corollary}
\newtheorem{obs}[thm]{Observation}
%\theoremstyle{definition}
\newtheorem{define}[thm]{Definition}

\lstset{frame=lines,framesep=3pt,numbers=left,numberblanklines=false,basicstyle=\ttfamily\small}

\include{macro}

\usepackage{todonotes}

\begin{document}

\title{ASlib: A Benchmark Library for Algorithm Selection}




%core group, cooperators; alphabetically ordered

\address[muc]{LMU Munich, Germany}
\address[muenster]{University of M\"unster, Germany}
%\address[cork]{Insight Centre for Data Analytics, University College Cork, Ireland}
\address[freiburg]{University of Freiburg, Germany}
\address[vancouver]{University of British Columbia, Vancouver, Canada}
\address[paderborn]{University of Paderborn, Germany}
\address[tue]{Eindhoven Institute of Technology, Netherlands}
\address[ibm]{IBM Research, United States}


\author[muc]{Bernd Bischl}
\ead{bernd.bischl@stat.uni-muenchen.de}

\author[muenster]{Pascal Kerschke}
\ead{kerschke@uni-muenster.de}

\author[vancouver]{Lars Kotthoff}
\ead{larsko@cs.ubc.ca}

\author[freiburg]{Marius Lindauer}
\ead{lindauer@cs.uni-freiburg.de}

\author[ibm]{\mbox{Yuri Malitsky}}
\ead{yuri.malitsky@gmail.com}

%%%%%%%%%%%%%%%%%

\author[vancouver]{Alexandre Fr\'{e}chette} % (?) - helped to reformat the satzilla data
\ead{afrechet@cs.ubc.ca}

\author[vancouver]{Holger Hoos} % discussion about specification
\ead{hoos@cs.ubc.ca}

\author[freiburg]{Frank Hutter} % discussion about specification
\ead{fh@cs.uni-freiburg.de}

\author[vancouver]{\mbox{Kevin Leyton-Brown}} % discussion and writing
\ead{kevinlb@cs.ubc.ca}

\author[paderborn]{Kevin Tierney} % discussion about specification - maybe data set
\ead{tierney@dsor.de}

\author[tue]{Joaquin Vanschoren} % related work and data set about ml
\ead{j.vanschoren@tue.nl}




\begin{abstract}
% Re-draft 1 of first sentence: too clunky and long
% Different algorithms exhibit varying performance when solving a particular
% problem instance. The algorithm selection involves choosing an
% algorithm from a set of algorithms that most effectively solves a particular %%% Note: I (KBT) am explicitly avoiding algorithm schedules here to avoid complicating the abstract
% problem instance. The algorithm selection problem is attracting increasing
% attention from researchers and practitioners in AI.
The task of algorithm selection involves choosing an algorithm from a set of
algorithms on a per-instance basis in order to exploit the varying performance
of algorithms over a set of instances. The algorithm selection problem is
attracting increasing attention from researchers and practitioners in AI.
Years of fruitful applications in a number of domains have resulted in a large
amount of data, but the community lacks a standard format or repository for
this data. This situation makes it difficult to share and compare different
approaches effectively, as is done in other, more established fields. It also
unnecessarily hinders new researchers who want to work in this area.
%s
To address this problem, we introduce a standardized format for representing algorithm selection scenarios
and a repository that contains a growing number of data sets from the
literature. Our format has been designed to be able to express a wide variety of
different scenarios. Demonstrating the breadth and power of our  platform, we describe a set of example 
experiments that build and evaluate algorithm selection models through a common
interface. The results display the potential of algorithm selection to achieve significant performance improvements \hh{across a broad range of problems and algorithms.}
%\note{FH}{The abstract does not address the relationship to competitions -- that's fine by me, I just wanted to point it out in case that got left out.}
\end{abstract}

\begin{keyword}
algorithm selection \sep machine learning \sep empirical performance estimation 
\end{keyword}

\maketitle

\input{introduction}
\input{related_work}
\input{specification}
\input{current_scenarios}
\input{eda}
\input{experiments}
\input{conclusion}

\bibliographystyle{elsarticle-harv}
\bibliography{local}

\end{document}
